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present invention 
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is also 

The inventiin 
convendonc 



relates to touch input devices for electronic devices. The present invention 
to touch screen devices, such as PDAs, mobile telephones or handheld computers, 
also relates to touch screens and more specifically to implementing a dual input on 
1 single-point output touch pads. 



screisis 



Touch 
holds the d 
certain 
However, c 
with two 
pressapou 



„ are used in increasing numbers in handheld electronic devices. Usually the user 
svice in one hand and uses the user intaiace of the device with the other hand. In 
^™ however, it mi^t be useful to allow the user to use the UI with both hands, 
iirent resistive touch pads do not allow multiple input If a user touches the touch pad 
^ i, the device handles this is an enor and assumes that the user actually intended to 
t'that is the middle point of a line that connects these two input points. 
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fii Lgers, 
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laptop 
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example oi 
Ctrl, etc) tl 
a drag & 
requires a 
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power or 
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theref 5re 



It is furtbe : 
to recogni: 



Method and device for recognizing a dual point user input 
on a touch based user input device 



i^iany electric devices that use touch pads for user input, such as PDA, mobile phones, 
ers and PC monitors. Typically all of them aUow only single point user entry on 
area, such as pressing a graphical icon, a menu item or a drawing with a pen or 
..ver. there is a raising interest to utilize dual ponit user input in special cases. An 
this kind of use is a device ihat has a QUERTY-keyboaid with special keys (shift, alt, 
. at must be pressed with another key. Another commonly used user interface feature is 
<rop -feature that is not possible with current touch pad technologies as it typically 
hift-key pressed down. 



com outers 



mmt 



Ho^ ^ever, 



ctually some touch pad technologies that are capable of detecting more than one input 
aneously, but these are expensive, require too much operating power, processing 
D lemoiy for a mobile device. 



t iput 



desirable to have a cheap touch based input device that can recognize a user uiput 
points. 



desirable to enable a conventional touch pad that allows only a smgle point user input 
;e mxiltiple point user input. 
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in|Ut 



In another 
input on a 
a single 
related sin^< 
the same 
preferably 
determinin; 
generating 
using said 



a first aspect of the present invention, there is provided a method for recognizing a 
ii^ut on a touch based user input device comprising the operations of receiving a 
to said input device relating to a first position, fonning a first position signal 
s aid first user input, receiving a second user input to said input device relating to a 
fonning a second position signal relating to said first input and said second 
..^^ on the basis of said first position signal and said second position signal, if said 
input has its source in a simultaneous dual point user input, generating a third 
on said first position and said second position, and using said first and third 
the coordinates of said dual point user input. 



: ^pect of the invention there is provided a method for recognizing a dual pomt user 
tuch based user inpul device, ^vherein said mpvX device is only capable of outputting 
position signal. That is the touch input device provides on every kind of mput a 
le position output signal, but there are different input situations possible that produce 
Output signaL The method comprises fomiing or detecting a first position signa , 
itoring said position signal, forming or detecting a subsequent second position signal, 
if said second position has its source in a simultaneous dual point user mput. 
third position by reflecting said stored first position at said second position, and 
to position and said third position, as the coordinates of a said dual point user input. 



By foxmini a first position signal elated to a first user mput to said input device, it is supposed 
that a sing) s point user input is detected on said touch based input device. 



25 Bypreferajly 



tiie input 
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storing 
it is knowi l 



Storing said fiist and third position signals, the position is made available, even if 
r oint has actually changed its position. Position signals can be stored m the form of a 
f or e.g. in the fonn of e.g. binary coded coordinate data. It may be noted that the 
of the first use input position can be performed by using a transient memory, as 
from persistent storage scope technology. 



opi nation 
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may have 
between 
35 simultane ras 



properties 
can be 



By preferJbly forming a second position signal that preferably differs fix>m said first posm«a and 
rel: ted to a subsequent second user input to said input device, an event is detected that 
been caused by a dual point user input or by a single point user input To distmgmsh 
tjie two possible user inputs, it is determined if said second position has its source m a 
,us dual point user input This detemiination can be perfomied by evaluanng the 
of the signal transition from the first to the second position signal. This determmation 
llased on a differentiation between a substantially continuous and a substantially 
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discontinuods signal transition from the first to the second positian signal, wherein a 
substantiall) discontinuous signal transition indicates a dual point user input and a substantially 
continuous s ignal transition indicates siugle-point user ii^ut, i.e. a motion of the input point on 
the touch ba^ad input device. 



poi it 



If a dual 
first positioi I 
used as 



user input is detected, a third position is generated by (point) reflecting said stored 
on or upon said second position. Said first position and said third position, are ^en 
cooijdinates of a said dual point user input 



10 Preferably 
input at saic 



stid generated third position is essentially the same location as the said second user 
second positioiL 



«flection operation of said first position at said second position visualizes the 
said third point. The criteria for a dual-point user input is fiilfilled, if said second 
the *cent« of mass' position of two actually pressed points on the touch 
inputjdevice. With center of mass information (second position) and one of two points (i.c. 
I, the third position can be calculated. 



The point 
generation 
position 
based 

first positioi), 



<if! 



represents 
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can also be obtained by generating a difference signal between the stored first 
the second position, and adding said difference signal to the achial second position. 

a signal-based generation of the third position, ft is siqiposed that a generation of 
dtion by calculating the position coordinates of the positions is easier to implement 



For the abokre reasons, a device using this method can distinguish between user-input cases with 
a single prt ssing point or a dual pressing point When the separation has been done, the method 
deterauneslwhere the second mprA point is. as the hardware then produces incorrect data. 



phrt of said method can be regarded as a static case, wherein the second point is not 
Tie presait invention can also be apphed, if a movement of the second point is 
By continuously reflecting die first point at the second (moving) point, a movement of 
pjint can be calculated. So the first point can serve as a reference point for generating 
of the third point. 



In another example embodiment of the present invention said determination, if said second 
position ha s its source in a simultaneous dual point user input, is based on the gradient between 
the first po sition signal and the second position signal which may be the gradient of the position 
signal fionf said first position to said second position. The gradient of flie position refers to die 
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time derivat ve of the position, and is proportional to the speed said point is moving. If the 
position sigr al rises up abruptly, the position signal becomes substantially discontinuous, and the 
gradient increases. A substantially discontinuous sigual transition indicates a dual point user 
input and a substantially continuous signal transition indicates single-point user input, e.g. a 
motion of a single input point on the touch based input device. Instead of the gradient, the 
steepness of the signal within the transition area may also be used as a criterion to decide if the 
transition is Aiscontinuous or not. 
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noted 



It may be 
10 To implement 
time period 
range below 
an input 



actUEUCor 



that the first position should be stored wfaile the position is substantially static, 
this, the first position may be stored in a transient memory, to be available after a 
characteristic for a discontintious signal transition. This tuner period can be in the 
1/10 second, ^iiich is the maximum estimated time required to set down a finger or 
(e.g. a pen) on tibe touch pad 



In yet another example embodiment of tibe present invention said method comprises storing said 
third positioi i. If said second position is stored, it can be used as a reference position to calculate 
of the first position if a motion of said second position is detected. 



a movement 



In another 
a motion 
of reference, 
reflecting 
implementat 
detected. As 
'center of 
second poim 
regarded as i lx 



of i aid 



s;id 



grivii 



ample embodiment of the present invention said method further comprises detecting 
second position, setting one of said first position or said third position as a point 
and calculating a motion of said position which is not said point of reference, by 
point of reference of said second position. This represents a dynamic 
on of the method in dual point input mode in case a motion of the 'middle' point is 
set forth above, the touch pad can only detect the motion of the middle point or the 
ry' of said dual-point user input. There is only one case in which a motion of the 
can be interpreted in an unambiguous way that is when one of tibie points can be 



To use one ix reference point, this reference point has to be stored- For input features as e.g, 
string, alt, c< ps lock and the like user input, the first position can be used as a reference point, as 
it can be ass uned that the position used to press a 'string' input area on the touch screen is not 
likely to be r loved. 

hi case of a drag-and-drop' user input, it is supposed that that a user first points to an object to 
be dragged, >iesses subsequently an input area to activate the 'drag and drop' function, and then 
moves the o >ject. In this case it can be assumed that the position used to activate the drag and 
drop featurq (i.e. the third position) is not moved on the touch screen, and therefore the 
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tlird 
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6^ . 



By 



In another 
a signal, 
reference 
device, both 
application, 
point input. 



ample embodiment of the present invention said mefliod further comprises receiving 
wUch indicates if said first position or said third position is to be used as a point of 
receiving an infoxmation e.g. from a software application running on said user 
kinds of input features can be in^lem^ted in a singe device or under a single 
In this case the spplication can decide on base of the actual positions of the dual 
f the first or the third point should be regarded as a reference point. 



It is also possible to principally select the pomt, which is positioned closer to the left side of the 
input device as the refcTOTCe point. It is also possible to principally select the point, which is 
actually pos tioned closer to the ligiht side as the reference point 



this 



By using 
to hold a t4uch 
perform an 
hand to tap 
left handed, 
the left sideps 
side can be 
achieved by 



By using be undaiy 
or to a not i 11 
ii5>ui. Then by 
dualuQ>ut 
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position can be used as a fix reference position. It may be noted that the setting 
point may be performed before a motion of the second position is detected. 



right/left side reference point ^proach it can be taken into account that users tend 
enabled device in their non-dominant hand, and use their dominant hand to 
] nput e.g, with a finger or a pen. A user can easily use the thimib of the non-dominant 
Dn the touch-input device. As it can be expected that a user is either ri^t handed or 
it can be expected that in case of a right handed user the point positioned closer to 
is pressed by tibie thumb. Therefore it maybe expected that the point closer to the left 
jsed a s reference point. Thus, a natural way to control touch input based devices, is 
the combined movements of two points, such as the pen and the thumb. 



In yet anothbr example embodiment said determination, if said second position has its source in a 
simultaneoi s dual point user mput, is based on boundary areas. The boundary areas are defined 
by possible input options and said first position. A dual point user input is excluded, if at least 
one of said lecond positions is detected to be outside of said at least one boundary area- 



areas, an input that shows a discontinuous signal but leads to a not acceptable 
Lterpretable second input signal can be excluded from being recognized as dual-point 
a number of possible input signals can be excluded from bemg recognized as a 
the beginning. 



f 'om 



In another txample embodiment said input area is defined by a 'half edge distance area' from 
said first pc sition. A 'half edge distance area' around the first point can define a basic boundary 
area. If the second input position is detected outside of the half edge distance area, the second 
point woul< . be calculated outside of the sensible area of the touch pad- So when calculating the 
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tie 



obta ned 



p( int 



lets 



excli ided 



noled 



It maybe 
therefore ma; f 
user input pi ^cti^ 
input is 
dual point 
areas, see 



redu :ed 



uj er 



figures 



In yet 
setting a * 
user input. 



anothtr example embodiment of the present invention said method further comprises 
da^ point user input flag% if said second position input has its source in a dual point 



use: ul 



It can be 
input mode 
send from a 
input device, 
thus can 



Even in the d lal 
a characterist ic 
being in con 
position vani 
is detected 
reference 
position or 



•th5 
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third point from a second point outside the half edge distance area, an invalid 
. To prevent that faulty third points can occur, the second point is regarded as a 
user input, if the distance between the first user input point and the second user 
too big- So a step longer than a usual one is interpreted as a single point user 
using the half width boundary area of a possible new second user-input positions 
firom a double point user input Therefore, the accuracy can be increased 



Aat the boundary areas may depend on the position of the first position, and 
have to be calctilated. The boundary area concept can also be regarded as a kind of 
ion, wh^ein the area in which a second use input is acc^ted as a dual-point 
By using boundary areas the reliability of the recognition and the operation of 
input can be significantly increased. For fiirther implementations of boundary 
9 and 10. 



if the device is capable of being aware if the touch pad is actually in a dtial point 
clr not. The method can also comprise a 'dual point user input enabled'- flag diat is 
iser ^plication, to enable and disable a dual point user iiq>ut on said touch based 
The flag can be used to add constraints to the recognition of dual-point input, and 
the accuracy of the recognition process. 



mcnase 



In yet anothei example embodiment of the present invention said method fijrther comprises using 
said second position as the actual position of a single point user mput, if said dual point user 
input flag is set and if it is determined that said second position input has its source in a dual 
point user uq ut 



point input mode the behavior of the movement of the second position can show 
discontinuous transition behavior, when the user lifts of one of flie two elements 
act with the touc^ pad. In this case the reference point or the 'calculated' ^rd 
ihes. If the calculated point vanishes, the calculated position or the second position 
(continuously or discontinuously) to the reference point Analogously, if the 
vanishes this is indicated by a 'jump* of the second position to the calculated 
calculated tjump* of the calculated position to the reflection of the reference point 



to retum \ 



poi it 
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position, 
new center 
generation o 
two points 



ted position. In this case the set flag can be de-set If none of diese two cases occur, 
move of the second position to a fourth position can be used to calculate fifth 
a third touch point on the touch pad. In this case it is to be noted that the 
gravity position effects requires a different set of calculation equations than the 
rthe third position^ to take into account that the second position actually represents 
not a single one. 



rep -esenting 



a] id 



The me&od 
method can 
10 detected 
actually not 



can further comprise de-setting ox re-setting of said dual point user input flag. The 
further conxprise de-setting of said dual point user input flag, if no user input is 
is the flag can automatically be de-set, if the touch pad detects that the user is 
\ ouching the touch pad. 



Thit 



that 
Ic 



According tc 
indication 
option may 
inpuL There 
input mode, 
perfomi diis 
Assistants 
would diere 
^reference 
touch pad. 
regarded as 
So even if a 
where the 
connection 



In another 
tide new 
30 not in a 



According tc 
from a 



server 
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another aspect of the present invention, the method further comprises displaying an 
the dual point user input is used. A user who is not aware of a dual user input 
astonished or even frustrated^ if the device reacts not in an expected way to a user 
ore it can be useful to indicate that the touch pad / screen is actually in a dual user 
An indicator, an inserted icon or a cursor displayed on a display of the device, may 
Cursors are actually not used in touch screen devices such as Personal Digital 
(I|D As), as the cursor would be positioned below the finger or the input actuator, and 
bre not be visible. In case of a dual point user inpxit, it may h^pen that die 
* is moved and so title cursor position can deviate from the contact position on the 
cursor can be used to indicate by its form, which of the two points is actually 
] eferOTce point A cursor can provide a clue why the device reacts in a certain way. 
user is not aware how a dual point input is generated, the user can easily recognize 
cursor is located in the view of the device. The cursor can be implemented as a 
between said reference point and said calculated point. 



pcinf 



a( ;tual 



Ine 



ecj 



ample embodiment said metitiod further comprises setting said second position as 
posiKon of an actual single point user input, if said second position input has its soxirce 
dual point user input. 



According td yet anoflier aspect of the invention, a software tool is provided comprising program 
code means Tor carrying out the method of the preceding description when said program product 
is run on a a mputo: or a network device. 



another aspect of the present invention, a computer program product downloadable 
for carrying out the method of the preceding description is provided, which 
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comprises 
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According 
comprising 
methods of 
network device, 



yet another aspect of the invention, a computer program product is provided 
]lrogram code means stored on a compute readable medium for carrying out the 
he preceding description, when said program product is run on a computer or a 



da a 



According t< 
computer 
computer pe 
computer pn 



another aspect of the present invention a computer data signal is provided The 
signal is embodied in a carrier wave and represents a program that makes the 
form the steps of the method contained in the preceding description, when said 
gram is run on a computer, or a network device. 



According tc 
controller foi 
outputting a 
comprises 



anodier example embodiment of the present invention a touch based input device 
a touch based user input device is provided. Said input device is only capable of 
single input position signal that depends on the actual user input. The controller 
input that is connectable to said touch based user input device, a metnory, a 
difrereatiato4 a first and a second evaltiation circuit and an output. 



ai. 



Said input is 
signals from 
restrictions 
irxput position 
supply the input 



wherein 



The memory ^s 
memory can 
as a referenc< 
moments, 
changed to a 
can provide 
pre-processiqg 
said position 



Said 
different 
transition timb. 
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8 



code means for performing all of the stq>s of the preceding methods when 
is nm on a computer or a network device. 



cotmectable to said touch based user input device, to receive successive position 
said touch based user input device which a user has touched. Because of the 
' die touch based user input device, tiie input can only receive a singe point user 
signal. The input can also be implemented as an interface to said input device to 
device with power- 



connected to said input, to store at least one of said received position signals. The 
dso be connected to one of said evaluation circuits to store a calculated position e.g. 
point. The memory is to be able to store a position signal at (at least) two different 
the need to store a first position is detected when the position signal has 
second position, and the first signal is not longer accessible. A transient memory 
1 his. The memory can directly be connected to said input or indirectly via a signal 
stage, such as said first or said second evaluation circuit. The memory can store 
signal as the signal itself or in a coded form such as parameters or coordinates. 



differedtiator is connected to detect time dependent transition properties between two 
following positions, to determine e.g. the time gradient of transition and/or the 
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following a 
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incorporated 
it is likely 



evaluation circuit is connected to said differentiator to det^niine, if a position 
preceding position is caused by a single point usct input or by a dual point user input. 

circuit can also be connected to said input The differentiator can be 
in said first evaluation circuit. The first evaluation circuit is provided to determine if 
' thbx dual-touch input is actually p^ormed or not 



Said second 
evaluation 
on the basi^ 
performing 
position. 



15 



Said output 
put cut saic 
application 
interfece to 



evaluation circuit is connected to said input to said memory and to said first 
circuit Said second evaluation circuit is provided to calculate a dual point user input 
of a first input position and a successive second position. This may imply 
he calculations required to reflect a first input position at a successive second 



connected to said second evaluation unit, and is connectable to a processing unit to 
calculated a dual point \iser input to a application device, for providing an 
^ ^th singe point and dual point inputs. Said output can also be implemented as an 
£ aid input device to be supplied with power by a connected {^plication device. 



In anotiiCT e? ample embodiment of the present invention said touch based input device controllCT 
further com arises an input connected to said second evaluation unit that is connectable to a 
processing v oit to receive control information from said processing unit to control the operation 
of said secox d evaluation unit The control infbrmation can comprise e.g. *dual input enabled*, or 
*first/second position is reference point', or e.g. boundary area related information. The input 
controller c£ n also be implemented integrally with a touch based input device such as a toxich 
screen modiue or touch pad module. The input controller can also be implemented integrally in a 
touch screen controller. 



t<^ 



According 
a touch 
device to 
according to 



bas«d 



Si id 



In another e±ami 
device can bb 



another aspect of the present invention an electronic device is provided compnsmg 
input device, a processor and input controller connecting said touch based iriput 
processor, wherein said input controller can provide a dual point user input 
the preceding description. 



Lple embodiment said electronic device is a mobile terminal device. The terminal 
embodied as a touch screen PDA, or a touch screen telephone. 



In the following, the invention will be described in detail by referring to the enclosed drawings in 
which: 
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Figure 1 dcAicts a two point input and respective touch pad output in case a of conventional 
touch based iser input device user interface, 

5 Figure 2 dep icts a track of a stylus moved on touch pad surface by a user. 

Figure 3 sho ivs the x-axis and y-axis signals caused by the movement of figure 2. 



Figure 4 de jicts 
10 resistive use : 



Figure 5 
point. 



vis lalizes a signal discontinuity caused by a user touching a touch pad at a second input 



the use of ihe signal raise time used as a judgement parameter between 
discontinuity or not situation. 



15 Figure 6 visualizes 



Figure 7 vis 



Figure 9 
the present 



25 Figure 10 is 
the boundai y 



Figure 11 s 
touch basec 



30 



It may be i 
figures are 
is 

P2. 



; represent ed by 



35 



Figure 1, 
pad that 
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10 



a two point ii^ut and respective toudi pad output iii case of a conventioiial 
inter&ce. 



lalizes the pnxjess of reproducing the conect position data of two input points. 



20 Figure 8 is j flow chart of an implementation of ttie method of the present invention. 



deiicts different embodiments of boundary areas of an implementation of the method of 
nvention. 



a flow chart of another implementation of the method of the present invention using 
areas of figure 9, and 



jhematically depicts an implementation of a touch based input device controller for a 
user input device. 



oted that the position points Pi, Pj and Pm used in the foUowing description of the 
represented by the first, second and third positions used in the text. The first position 
P,, second position is represented by Pm and ibe third position is represented by 



S LOWS 



US»1 



an input on a conventional electronic user input device such as a resistive touch 
by devices such as PDAs, mobile phones, laptop computers and PC monitors in an 
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pom :s 



illustrative 
allows only 
with a pea 
pressed 
a single usei 
and p2 are p 
single point 
the i 

incorrect sij 



ttuch pad having a lOx 10 matiix. Typical to all of them is that user input area 
single point user entry, such as a pressing a graphical icon, menu item or drawing 
stylus. The resistive touch pad hardware behaves in a way that in a case of two 
the resistive properties of the input area converts the input into a signal indicating 
input point in the middle of the actual user input points. When the two points Pi 
essed on the active input area, a conventional toudi pad. (which is designed for 
entry) interprets the situation so that only one point Pm is pressed in the middle of 
line between these two points. Therefore the hardware produces actually an 



mterconi lectmg 



gial. 



In figure 2, t 
fix>m a certa] a 



Jn figure 3, 
Figure 2 are 
slow, a fast 
the signal 



the X-axis and y-axis signals caused by the movement along the track depicted in 
shown. The different output signals represent different stylus moving speeds for a 
1 ind a very fest movement of the stylus (ftom left to right). Although the speed varies 
continuous, and no discontinuities occur. 



ienams( 



Figure 4 
user interfac e. 
first point Ii 
linebetweei 



esi< 



There are 
1) detectinj 
2) 



calculati ig 



These phas4s 
coordinates 
figures 5 
calculating 



Figure 5 
a user toud 
second 
the time a 
pressure is 



poitt 
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user is moving a stylus over a touch pad surfiice. In the example the stylus is drawn 
start position Xstaitj Ysim to an end position XEnd? YehO- 



dei lets 



a point input and respective touch pad output in case of conventional resistive 
The pressing of first point Pi followed by a pressing of point P2 is interpreted as a 
is pressed followed bay a pressing of point Fm in the middle of the interconnecting 
Pi and P2. 



Initially two phases that are used in dual point input detection and input: 
a dual point input (as separated from normal smgle point input), and 
the second real user ii^ut point 



can be used to implement the dual point user input and to produce two pairs of 
for these ixrpvt points, which can then be used in UI applications. In the following 
6 are related to the detection of a dual point input, and figure 7 is related to 
1 he second real user input point 



aiid 



dei icts 



a discontinuous signal or a signal discontinuity caused by a second user input i.e. 
;l|ing said touch pad at a second point Hie signal changes very quickly in case that a 
on the touch pad is pressed. The signal transition time is primarily determined by 
slylus or a finger needs fix)m the first contact of the touch pad surface, until a certain 
)uilt up. This time period can be estimated to be significantly below e.g. less than a 
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1/10 of a se »nd. Compared to a typical stylus move, that which can be expected to require a 
time in the r inge of a few 1/10 of a second, the bolh signals can be distinguished. Therefore the 
signal rise lime can be used as judgement parameter between a continuity situation and a 
discontinuity situation. 

5 

Figure 6 dej icts a discontinuous signal raise time, in an enlarged time scale. The discontinuity 
evaluation (an be appUed to both X- and Y-coordinate values. It is sufficient to detect a 
discontinuit:' in one of the coordinates. In case that e.g. the point Pi and P2 have the same y 
coonJinates a discontinuity can only be detected in the x^ordinatc, and vice \fxs&. In the 

10 depicted diagram the discontinuity can be described by two parameters, Ae signal rise time or 
transition tii le Ato and by the gradient of transition S. The gradient of transition is proportional to 
the position change po divided by transition time Atb. The larger the change is, the larger is the 
gradient of transition S. Both values can be appUed to detect a discontinuity. Using only the 
transition til ae Ato can lead to a situation in which a smaU position change (e.g. one digit) may be 

15 recognized as a discontinuity. The gradient of transition S has the advantage thai for small 
positicm chj nges can automatically be regarded as continuous. 



be lavior 



The dual po int 
typical toucji 
where user 
method hov ' 
domain 
hardware ■w 
interpreting 
actuator (b; 
hardware s 
remains 
this signal 
other input 
signal char 
handling/pi ocessing 



can be 
analysis 
be used in 
slope 
based on 
Basically 



user input can be detected with the foUowing procedure. As it was mentioned, the 
pad hardware produces a single input point data in noimal use and also in a case 
presses two points. In order to be able to utiUze the dual point input there must be a 
to separate these two cases from each other. This can be done via analyzing the time 
of the hardware signal. In typical noimal use, as user presses touch pad 
ith an input actuator (such as finger, stylus or pen) and therefore produces a signal 
the pressing point. The iiqmt point can also move while user is dragging the input 
' sliding, drawing, scrolling a menu, etc). In all of these normaVtypical cases, the 
gnal is continuous (see figure 3). The movement might be very fast but the signal 
continuous. However, when an user touches the touch pad at a second position, 
ej^eriences an instant and very rapid discontinuity indicating that there must be an 
point present (see figure 5). This knowledge can be utilized by setting a limit for the 
ge rate. The signal change rate is an expression that is common to electrical signal 
art and describes the increase or decrease time of a si^ial. The change rate 
by signal edge detection, a Schmitt trigger, a high pass filter or by Fourier 
high frequency component detection. The determined signal change rate value can 
ju(^g if the input is made with single or dual presses. If the signal exceeds a given 
the discontinuity is detected. The proper value for the limiting factor can be set 
usability smdies so that he use of dual input touch pad is convenirat and namral. 
his is only a question of finding a feasible value for the limiting factor that is 



ah^ays 



del Tmined 



wth 



steei ness 
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compatible 
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vithi 



the natural way of humans iising touch pads. The described process is illustrated 
in the figures 8 and 9. Naturally, this elementary process must be applied 
sequentially luring iiq)ut activity in order to have a continuous detection method. 



flowcl art 



visilalizes the process of producing correct position data of two input points. When two 
p essed on active input area, the device (which is designed for single point entry) 
situation so that only one point is pressed in the middle of the interconnecting line 
two points (see figure 1 for illustration). If a two point input is detected, the first 
pressing poiiLt and flie "faulty middle point" is known which is enough information to calculate 
the actual sei^ond pressing point as explained below: 



Figure 7 
points are 
interprets the 
between thege 



Pi = {Xi.Yi 

P2-{X2.Y2 

Pm {Xm,Y 



From ^ese 
by... 



Thus, the ft 
calculated 
input points 



The followi ag 
Enabling th 
combinatioqs 
generahzed 
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a} 



first actual and detected user input point with coordinates 
second actual user input point with coordinates 
second detected user input point with coordinates 



As user is j pressing two points, we know the first (and previous) pressing point Pi and the 
incorrect rai< die point Pm, which is produced by faulty hardware interpretation of the actual input 
actuation. T< gether with the detected dual point input case (as explained in figures 5 and 6) there 
is enough in brmation to calculate the actual second actual pressing point. First, the middle point 
Pm for any ^ i^o points Pi and P2 can be defined by. . . 



i quations the correct actual position of the second user input point P2 can be derived 



X2 = 2Xid''X\ 



st user input point Pi is known and the second actual user input point P2 can be 
on misinterpreted touch pad signal. Therefore correct data of a dual point user 
are available for user inter&ce applications. 



b ised 



table illustrates the correct one to one relationship between Pi, P2 and Pm- 
s calculation method to be used for any pair of user input points and with any 
of relative positioning of fhem. Therefore the presented simple idea can be 
o any size and shape of the touch screen displays or otiier touch sensitive areas. 
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Matrix no. 



Vettcal 



Hon sontal 



First position Pi 
X Y 



1 
2 
3 
4 
5 
6 
7 
8 



second position P2 
X y 



2 
2 
8 
8 
2 
2 
2 
8 



2 
8 
8 
2 
2 
8 
8 
8 



middle point Pm 
X Y 



8 
8 
2 
2 
2 
2 
8 
2 



8 
2 
2 
8 
8 
2 
8 
8 



5 
5 
5 
5 
2 
2 
5 
S 



5 
5 
J 
J 
5 
5 
.8 
8 



It may be n >ted that the positions can conq)ri5e more than one possible user input point, as the 
equations nay lead to non-integer position values. The non-integer values may be avoided by 
interpolatin; 5 the position values or by using a touch area instead of a second position. The 
position resalution of the second point is decreased, as the positioning error of the calculated 
thir d point 1 '2 is increased by a fector of 3. 

The dual pc int user input can be used fbr new user inierfeoc features such as two item selection, 
shifl/alt/ctri functionaUty in on screen keypads, drag & drop, keyboard shortcuts, etc... in the 
case when resistive touch pad technology is used. The operation principle is simple and 
implement? tion requires only smaU modification into software (hardware driver module). The 
invention can also be implemented in a hardware module. The present invention allows the 
implements tion of new user interface styles. 

If the dual >oint input is activated and the middle point Pm is moved, the one to one relationship 
is not long<T existent If e.g. the middle point moves one step to the right, it can principally not 
be detennii ed if the user has moved each point a single step to the right or one of them two steps 
to the right In some cases it is however possible to deteraiine which was tiie actual user input . 

One possit LUty resides in that ahvays the first point is used as a fix reference point to calculate a 
movement of the second point accoitJing to the above equations. This possibility is very useful at 
the sbift/al/ctrl ftmctionaUty, in on screen keypads, keyboard shortcuts, and all applications in 
which the 1 irst position is supposed to be stationary. 
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In case of tie diag and drop feature, it is expected that a user first points to item and then 
activates the drag functionaUty by pressing a second point on a touch pad or the touch screen. In 
this case, tie calculated second point is supposed to be stationary. In contrast to an ad hoc 
approach 1h( : calculated second point is fixed and the motion of the first point can be calculated 
fiom the mc vement middle point. This may simply be implemented e.g. by exchanging the first 
and second >oints before setting the reference point and calculating the movement. 



Figure 8 is 
starts with 
rate is 
change rate 
one-point 
the surface 
said touch 



deter nined. 



flow chart of an implementation of the method of the present invention. The method 
iie detection of an input event at the position P,. In the next step the position change 

e.g. by determining 82 if the change rate exceeds a predetermined value. If the 

does not exceed this value the change is regarded as a conventional motion of the 
input at a point P,. This is the case if ihe point Pi remains static or is moved over 
of die touch-input device. The point P, is then reported 84 to die appUcation using 
j nput device as a user interfeice. 



user 



If the Chan ;e 
or a 'jural 
detected Si 
point Pa 
generated 



. rate exceeds the tiweshold value, the change is regarded as a discontinuous motion 
' of the one-point user input This if the jump is detected, a new input event is 
at die point Pm- The points Pi and Pm are tiien used to calculate 90 a second input 
aialogue to die above equations. The new double or dual input points {FM are 
i>2 and reported 84 to tiie s?>plicatioii using said touch input device as auser interface. 



pou it 



Figure 9 
a two 
exclude a 
25 examples 



matrix 



In the 
jimip to 
detected 
not inside 
dual-poin 
defined 
having 
half-edge 
boundary 
inputs 
tiiana 



depicts examples of how boundary areas can improve the accuracy of the recogmuon of 
. user input on a touch-mput device. Boundary areas can be defined and used to 
imnber of falsely recognized two point user inputs. In figure 9 there arc four different 
)f boundary areas indicated in the 10x10 input matrices numbered 1 to 4. 



usial 



^ number 1 the point P, is positioned at near the lower left comer. If a ascontinuous 
point Pm is detected, the point P^ can easily be calculated. If die point Pm is instead 
^. at the position of P2, a respective calculated point would be positioned outside and 
the matrix. To prevent that the calculated points are positioned outside the matrix, a 
input may only be detected if the new point Pm is detected within a boundary area 98 
the 'half edge distance' lines 94. The half edge distance lines 94 represent all points 
^ distances to the edges of tiie touch pad and the first point Pi. A combination of all 
distance Unes 94 represent die boundary 96 of the boundary area 98. By using a 
area 98, tree quarters of ttie input area and therefore diree quarters of the possible user 
be excluded fiom being recognized as possible dual point user input. A jump longer 
one (beyond the boundary area 98) excludes a dual point user input. It is to be noted 



tie 
eg 



b^ 



ecual 
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that the posil ton of this boiindaiy area d^ends on the position of the first point Pt and may have 
to be calcula ed. 



cont itn 



In the matrL: 
borderline 1 
98* can 
and the like 
shift/alt/ctrl 
area 98'. The 
that that righ 
thumb to 



I pre IS 



number 2 the point Pi is also positioned to be near the lower left comer. The 
0 separates the border area 98* form the rest of the touch pad area. The border area 
user inter£a<ee features such as the shifl/alt/ctrl functionality, keyboard shortcuts. 
The border area 98* can be used as a boundary area for the point Pu when 
unctionality, keyboard shortcuts input areas (not depicted) are located within said 
boundary area 98* can be used for e.g. right-handed persons, wherein it is supposed 
handed person uses his non-dominant left hand to hold the device and uses the left 
the shift/alt/ctrl functionality, while the right hand wields an input pen. 



In case of a 
located on 
100*. In a 
contents of 
optimized 
settings / use : 



l3fl] 



handed person said shift/alt/ctrl fimctionality input areas should be (analogously) 
right-hand side of the touch-input device. This is indicated by the interrupted line 
ijreferred embodiment the electronic device offers a possibihty to 'reverse* the 
.g. a touch screen display to enable left-handed persons to use the device in an 
The left-hand right-hand reversal may be implemented in a user selectable 
profile menu. 



th3 



wiy. 



In the matrix 
and combine 
matrix 
within the 
different 
accuracy 



numbsr 



ai^a 



In the matri> 
feature as 
from drawinj ; 
to the point 
onto tiie tou( 
input areas 1 
of the input 
enabled if 
movement o: ' 
points to 
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number 3 the n^t hand borderline 100 separates die border area 98' for point Pi 
it with a half edge distance area 98, defined benveen the lines line 94 and 100. The 
3 enables the recognition of a dual point input only when the pomt P| is located 
98* and when the point Pm is located within the area 98. That is there are two 
based constraints to enable an dual point user input, which in tuin increases the 
of a| the recognition of a dual point input. 



pos tion 1 



koDwn 



number 4 there are different input areas 102 provided representing each an input 
firom drag & drop-feature or the activation of different input styles as known 
programs. The input areas 102 can e.g. define a drawing- or an eraser-fimctionality 
i actually touched by a pen. Assuming that at the point Pi an input actuator is set 
h pad before an input on one of the input areas 102 is expected. When Pi and the 
32 are known and the only perfoimable dual-point iirput coniprises an input to one 
ireas 102, the boundary areas 104 can be calculated. Dual-point input is then only 
when a discontinuous jump into one of the boundary areas 104 is detected. If a 
point Pm is detected, the point P2 within the input areas 102 are used as reference 
the movements of Pi from the movements of Pm- 



aid 



calc ulate 
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compared 



wi h 



input predict 

I may be not 
areas 102 ar( 
side of the 



Figure 10 is 
10 Basically the 
steps are not 



T-284 S. 021/035 F-355 

PCT/IB03/03605 



17 



number 4 the number of possible dual point inputs are considerably reduced as 
the conventional methods. The boundary areas 104 can be regarded as a kind of 
on used to increase recognition accuracy of dual-point inputs. 



d that the matrix 4 is embodied as a matrix for left handed users wherein the input 
operated by e.g. the thumb of the right hand« and therefore are located at the right 
4, 



matrix 



a flow chart of another implementation of the method of the present invention, 
method comprises the same steps as disclosed in figure 8, and therefore &e similar 
described, but reference is made to the description of figure 8. 



The method Jiffers 6x>m the one disclosed figure 8 by an additional inquiry step 1 1 inserted after 
the detectioi 80 of an input event at ttie point Pi, to determine if the input event is detected 
within a bo mdary area. If the input event is not detected within said boundary area, it is 
presumed ttat the input is not caused by two-point user input, and tihat a single input is 
performed 8 I at the new single input point. 

If the input i ; detected within a boundary area, the second input is detected 88 at the point Pm and 
the method i roceeds as described in figure 8. 



The present 
boimdary 



ari^ 



depicts 



Figure 



Figure 11 
device 

touch screei , 
figure furthqr 
user ixsput 
interface 16 
evaluation 
signal j5x)ra 
speed at 
the 

predetermin sd 
point user 
to the 



bgi* 



which 



evaluBt on 



secoid 



method can further comprise steps like determining input areas and calculating 
to speed up the process. 



schematically a touch based input device controll^ for a touch based user input 
1 1 comprises three blocks, a touch based input device 2 such as a touch pad or a 
a touch pad input controller 6, connected via an interface 4 to said touch pad 2. The 
comprises a processor 18 for running an application, which is to be controlled by 
da said touch pad 2. The controller 6 is connected to the processor 18 via an 
The controller 6 comprises a memory 8, a differentiator 10 and first and second 
c 12 and 14. In tiie controller 6 the differentiator 10 receives a single position 
The touch pad 2 and determines the time derivative of the position signal, i.e. the 
the signal is moving on said touch pad 2. The determined value is transferred to 
circuit 12, to determine if the change of the position signal exceeds a 
limit If the limit is exceeded the signal is regarded as discontinuous, and a dual 
is identified. The information that dual-point user input is present is transfenred 
evaluation circuit 14. The differentiator 10 and the evaluation circuit 12 are 



ii Lput 
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provided to i letCTtnine if dual-point user input occurs or not. If dual-point user input is detected, 
the second e valuation circuit 14 is used to determine the two actual positions at which a user is 
expected to ijouch said touch pad 2. 



The second 
acmal 

calculate bo 
foregoing 
the calculated 
apphcation 



( valuation circuit 14 uses a fbnnerly stored first position stored in memory 8 and the 
received via the interface 4 to calculate an actual dual point user input. To 
h positions of an expected actual dual-point user input, the equations listed in the 
[cation regarding figure 7 can be used. The second evaluation circuit 14 transfers 
dual point user input via the interface 16 to the processor 18 to control an 
ijaiming on said processor. 



positi on 



specific 



The applicdtion running on said processor 18 may transfer control information via the 
inter&ce 16 :o the second evaluation circuit 14. 



inver tion 



In this i 
information 
resistive 
First, a dual 
actual 



The present 
designed foi 
on existing 
interface 



impleinenta ion 



resistive toi 
useful property 
input area, 
oiJy one pc 
Basically, c 
pressed 
case the 



inp it 



The operati >n 
the software: 
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the behavior of touch pads that are capable of outputting only a single position 
notwithstanding the number of actual input points or areas, as in the case of e.g. 
touph pads is used to allow dual inputs. The invention is essentially a two-step process- 
input situation is detected by monitoring the hardware signal. In the second step the 
point is calculated on the basis of the first input point and the middle point. 



secon 1 input ] 



invention provides a simple method to allow dual input on touch pads that are 
single input only, and provides therefore che^ possibility to implement dual input 
ouch based iiq>ut devices. The present invention allows for the creation of new user 
, that further improve usability of touch pad or touch screen enabled devices. 



fes tures, 



The metho^l is based on novel way of resistive touch pad signal inteipretation and the 
can be made with software. Therefore, the innovation can be implemented with 
ch pad devices or witti any other touch pad technology that behaves similarly. One 
of suitable touch pad technology is that when two points are pressed on active 
he device (which is designed for single poiai enny) interprets the situation so that 
int is pressed in the middle of the interconnecting line between these two points, 
nly an unambiguous signal and an unambiguous relation ship between a single 
point and two simultaneously pressed input points are actually required. In such a 
derivation of the third point P2 may be more complicated. 



principle is simple and the implementation requires only small modifications in 
of a hardware driver module. The performance or quality of the new feature is easy 
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to validate ai id therefore flie development time in research and development is short. 



c£n 



The present 
specific app 
invCTtion 
processing 
redesigned i 
user interfac ^ 



nvention can easily be implemented and tested. The present invention can be used in 
ications if the total user interface-style integration takes more time. The present 
be implemented simply by software and does not require significantly higher 
>ower or memory. The present invention allows for new input concepts and 
ser interface styles. The present invention allows the use of previously impossible 
features with dual point user input while utilizing existing hardware technology. 



It may be 
rectangular 
shaped touc|i 
even 
term can 



I $pheri< al 



Eoted that the present invention although described only in the case of plane and 
ouch input devices can also be appUed to round, oval or e.g. drcular or ring sector 
input devices. It is also possible to implement the present invention in a curved or 
touch input device. In case of a non-euclidic touch sensor distribution, a conrector 
to implement the present invention. 



be used 



It may also 
denote any 
displays, 



be noted that throughout the whole description the expression touch pad is used to 
kind of touch based input devices such as touch pads, touch screens and touch 



furdier 



It may 
than two 
discontinucius 
user-input 
unambiguo 
three-point 
middle 
subsequent 
(PC) user 



poi at. 



t> 



(an 



This application 
invention 
present indention 
invention 
the 

restricting, 
the enclos^ 
by the claipis, 



invent on. 
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be noted that the present invention can also be apphed to the detection of more 
user-input points. Starting from a two-point user input, and in case a second 
signal tr^ition is observed, the first middle point can be used to calculate third 
I ►oint on the touch pad. A problem arising from said three-point input resides in a not 
IS relation between a potential movement of the middle point of three points. In a 
input it is not clear which of the tree point acraally caused a motion of the actual 
:. But also there are some exceptions, a three-point user input such as can be a 
pressing of combination such as * String- Alt-Del' known to any personal computer 
restart Ae PC- 



contams the description of implementations and embodiments of the present 
the help of examples. It will be appreciated by a person skilled in the art that the 
is not restricted to details of the embodiments presented above, and that the 
also be implemented in another form without deviating from the characteristics of 
The embodiments presented above should be considered iUusttative, but not 
Thus the possibilities of implementing and using the invention are only restricted by 
claims. Consequently various options of implementing the mvention as determined 
including equivalent implementations, also belong to the scope of the invention. 



29/08 '03 VEN 14:54 [N** TX/RX 6535] 



29-AIJG-03 14:56 



Von-Pat8ntanwa6lt8 Becksr Kurig Straus 



+49 89 746 303 11 



T-284 S. 025/035 F-355 

PCT/IB03/03605 



5LO 



Claims 



1. Metihod 



for recognizing a dual point user input on a touch based user input device. 



10 



20 



25 



35 



compnsi ig 



- recci wng 

- foma ing 

- recei vong 

- fonr ing 

- determining 
said 

- gendraring 
sign 



inpit 



15 2. Method 
locatioT 



3. Methoc 
betwee i 



a first user input to said input device relating to a first position 
a first position signal relating to said first user input, 
^ a second user input to said input device relating to a second position, 
1 second position signal relating to said first input and said second input, 

^ on the basis of said first position signal and said second position signal, if 
second user input has its source in a simultaneous dual point user input. 
.....^ a third position based on said first position signal and said second position 
11, and using said first and third positions, as the coordinates of said dual point user 



accoiding to claim 1. wherein said generated third position is essentially the same 
as the said second user input al said second position. 



rding to claim 1 or 2, wherein said determination is based on the gradient 
the first position signal and the second position signal. 



acco: 



4. Metho4 according to anyone of claims 1, 2 or 3, fuitiier comprising: 
stoi ing said first and third positiorjs. 



5, Metiioi 

- det xting 

- set ing 

- cal :ulating 



pont 



30 6. Methoi 
positic 



according to claim 5, flirther comprising receiving a signal indicative if said first 
n or said third position is to be iised as a point of refeiencc. 



7. Mefhcd 
secoml 
boimc ary 
user 



accoiding to anyone of the preceding claims, fiirther comprising 
cting a motion of said second position, 
g one of said first position or said third position as a point of reference, and 
lating a motion of said position that is not said point of reference, by reflectmg said 
of reference on said second position. 



according to anyone of the preceding claims, wherein said determination, if said 
position has its source in a simultaneous dual point user input, is based on at least one 
area defined by possible input options and said first position, v^erein dual pomt 
iLputs are excluded if said second position is not detected to be within one of said 
bounc ary areas. 
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8. Method 
said first 

5 9. Method 



user iTspx it 



ccording to claim 7. whereiii said boundary area is a half edge distance area from 
position. 

recocts to anyone of the preceding claiins. fiufeer comprising setting a dual point 
flag, if said second position input lias its source in a dual point user mput. 



10 



10. Method 
- using 

poin; 
ina 

11. Me^od 



iccording to claim 9, further compnsmg: 
said second position as the actual position of a single point user input, if said dual 
user input flag is set and if it is determined that said second position has its source 
simultaneous dual point user input. 



according to anyone of the preceding claims, further comprising displaymg an 
indication that the dual point user ii^ut is used. 



15 



12.Methoc 
- setting 



second position 



20 13.Metbo<, 

only oi tputtmg 



14. Methol 
positic a 



25 



15, Methol 
differi 

16. Softw 



30 



canyt ig 
comp^ Iter 



mediion 



35 



isnic 



according to anyone ofthc preceding clahns, farther compnsmg: 
;aid second position as the new position of an acmal single point user input, if said 
input has not its source in a dual point user input 



according to anyone of the preceding claims, wherein said input device is capable of 
a single input position signal that depends on the actual user mput. 



according to anyone 
signal. 



of the preceding claims, &rther comprismg storing said first 



IS 



according to anyone of the preceding claims, wherein said second position 
ig firom said first position. 

»ol oomprid^ program code m«.B «»r»l » . compute readable M=diumJ6r 
ou. the meftod of anyone of claims 1 » 15. v*cb said software tool >s run on a 
or network device. 



17 Comdutcr program prod»cr con,prisiBg program code mears stored on a computer readable 
7. Com^u..^ ^ P« 3„y„^ of daints 1 .0 15. wh«. said program product 

on a computer or network device. 



18. Comimter program product comprising 



program code, downloadable fiom a server for 
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>at the method of anyone of claims 1 to 15, when said program product is run on a 
or networic device. 



compute 



19. Touch bised input device controller for a touch based user input device, , comprising, 

an input coimectable to said touch based user input device to receive successive position 
each representing a position on said touch based user input device, which a user 
t^)uched, 

; connected to said input, to store at least one of said position signals, 
differentiator to detect time dependent transition properties between two different 
positions, 

evaluation circuit connected to said differentiator to det«minc, if a position 
foUoiving a preceding position is caused by a single point user input or by a dual point 



signajls 
has 
am 
a 



succ< ssive ] 



a firrt 



user 
a 



nput. 



second 



circuit. 



iixpui 
an Q} 



evaluation circuit, connected to said ii^iut, said memory and said first evaluation 
wherein said second evaluation circuit is provided to calculate a dual point user 
on the basis of a first input position and a successive second position, and 
Atput. connected to said second evaluation unit, connectable to a processing unit. 



20, Touch 
coimecidd 



21. Electronic 
connect] ng 
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tased 



input device controller according to claim 19, further compiisiag, an input 
to said second evaluation unit, connectable to a processing unit to receive control 
infomiai|ion from said processing unit to control the operation of said second evaluation unit- 



device comprising a touch based input device, a processor and controller 
said touch based input device to said processor, characterized in that said 
controll<Jr is a controller according to claim 19 or 20, 



22. ElectroE ic device according to claim 21, wherem said device is a mobUe terminal device. 
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There is 
device, 
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said input 
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Further thert 
device, 
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device to 

15 user input 
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20 said memor 
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whei sin 



tie 



a method for recognizing a dual point user input on a touch based user input 
receiving a first user input to said input device relating to a first position, 
potion signal relating to said first user input, receiving a second user input to 
„ relating to a second position, forming a second position signal relating to said 
said second input, deteimining on the basis of said first position signal and said 
signal, if said second user input has its source in a simultaneous dual point user 
a third position based on said first position and said second position, and using 
third positions, as the coordinates of said dual point user input, 
is provided a touch based input device controller for a touch based user input 
. said input device is only capable of outputting a single input position signal that 
actual user input, comprising, an input conncctable to said touch based user input 
. successive position signals each representing a position on said touch based 
J, which a user has touched, a memory, connected to said input, to store at least 
position signals, a differentiator to detect time dependent transition properties 
. different successive positions, a first evaluation circuit connected to said 
to determine, if a position following a preceding position is caused by a smgle 
or by a dual point user input, a second evaluation circuit, connected to said mput, 
and said first evaluation circuit, wherein said second evaluation circuit is provided 
a dual point user input by reflecting a first input position at a successive second 
an output, connected to said second evaluation unit, connectable to a processmg 



re :eive 
d( ivice- 



iiput 



25 (Fig. 1) 
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